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(54) Title: VIRTUAL MAKEOVER 

00 

S (57) Abstract: A method and apparatus is provided for virtual colour makeover of a customer's face. A digital image is taken of 
^ the customer's face. Natural skin colour is determined by a differential analysis among at least two different sites along the face 
^ t0 identify an area without colour. The identified area without colour is then used as a basis for projecting the customer's face 
_ with the natural skin colour. Consultant's choice of pre-programmed colour palettes matching the measured natural skin colour is 
then projected on the facial image. A customer's personal choice such as fashion, natural or go-to- work looks can be registered 
for modifying the consultant's choice colour palette. The selected colour palette can then be identified as a set of colour cosmetic 
products, which are provided to the customer. 
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VIRTUAL MAKEOVER 

Field of the Invention 

5 

The present invention relates to a method and apparatus for 
allowing a consumer at point-of-sale to evaluate colour 
cosmetic products on their virtual facial image prior to 
purchase . 

10 

BACKGROUND OF THE INVENTION AND PRIOR ART 

Colour cosmetics encompass among others lipsticks, eyeliner, 
blush, lipliner, eye shadow and facial foundation. A vast 

15 array of colours and shades are available to consumers. 
Often the consumer finds it difficult to select among this 
vast array of choices. It is difficult to decide how a 
particular colour or shade will look when actually spread on 
the skin. Even more difficult to assess is how a 

20 combination of colour cosmetic products such as facial 
foundation, lipstick and eye shadow will work together for a 
particular individual. 

Systems have been developed to assist the decision process. 
25 Clinique and Clarion have installed computers at sales 
counters for use by the consumer. Information on colour 
shade, oiliness and other properties of a consumer's skin 
are punched into the computer which then determines the 
company's most closely matching product. 

30 

Another point of sale technique has been that of custom 
blending. Two major companies,' Prescriptives (division of 
Estee Lauder) and Visage (division of Revlon) begin a sale 
by manually evaluating a subject's skin colour. The sales 
35 person then adjusts existing finished foundations so as to 
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match the evaluated skin colour for a perfect facial 
foundation. Unfortunately, the system is time consuming, 
requires extensively trained sales persons, and has poor 
reproducibility . 

U.S. Patent 5,854,850, U.S. Patent 5,825,941 and U.S. Patent 
5,687,259 all to Linford et al . assigned to the Mirror 
Software Corporation disclose a system for digital image 
capture of a particular person's face and software to 
manipulate facial structures. The system is intended for 
reconstructive plastic surgery. 

It is evident that relatively unsophisticated systems are 
available for colour matching. • The technology also exists 
for digital recording of individual faces. To date there 
has been no disclosure of systems, which utilise digital 
imagery for cosmetic products. Neither has there been 
reported any systems for obtaining virtual makeover of a 
consumer's image whereby the consumer can see the effect of 
different colour cosmetic products on their own face. 

Accordingly, it is an advantage of the present invention to 
provide a method and apparatus for forming a virtual 
makeover of a person's face on an image monitor so as to 
allow selection of the best colour combinations. 

Another object of the present invention is to provide a 
method and apparatus for virtual makeover at a point of sale 
counter wherein colour makeup can be electronically painted 
over the consumer's image without requiring actual removal 
of make-up prior to obtaining a baseline natural skin colour 
image . 
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These and other objects of the present invention will become 
more apparent from the following summary and detailed 
description. 

5 SUMMARY OF THE INVENTION 

According to a first aspect, the present invention provides 
a method for applying a virtual makeover to a person's face, 
the method involving: 

(i) directing a digital camera at a person's face 
to register an image of at least a portion 
thereof; 

(ii) calculating colour parameters on two or more 
areas of the image to identify a natural colour 
of the skin; 

(iii) transmitting the image to a monitor for 
displaying the face; 

(iv) correlating the natural skin colour with a 
predetermined palette of colours appropriate to 
the calculated natural skin colour; 

(v) locating areas of the face for application of a 
colour cosmetic; 

(vi) displaying the facial image with the 
predetermined colour palette on the located 
areas; and 

(vii) optionally placing an order for colour 
cosmetics corresponding to the predetermined 
colour palette. 

30 According to a further aspect, the present invention 
provides apparatus for applying virtual makeover to a 
person's face, the apparatus including: 

(i) a digital camera for acquiring information on 
35 the person's facial image; 
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(ii) a program for calculating colour on two or more 
different areas of the facial image not 
normally covered by cosmetics to identify a 
natural colour of the person's skin; 
5 (iii) a monitor for displaying the facial image; 

(iv) a program for painting a predetermined colour 
palette correlated with the natural skin colour 
onto certain located areas of the face normally 
subject to colour cosmetic treatment; and 
10 (v) a printer delivering a printout of the 

predetermined colour palette. 

DETAILED DESCRIPTION OF THE INVENTION 

15 A problem with prior attempts to visualise colour cosmetics 
for customers has been the inability to determine their 
natural skin colour. Now it has been found that natural 
skin colour can be determined via digital imaging by 
comparing L*a*b colour values on at least two different 

20 areas of the facial image. Consumers do not place makeup 
over 100% of their face. The digital scan identifies those 
areas of skin which are not covered by makeup, provides a 
colour value for that area and utilises the value as a 
baseline colour for re-imaging a cosmetically stripped face 

25 onto a monitor. 

Another challenge for the system was to locate those areas 
of the face upon which colour cosmetics can be applied 
electronically. The program locates colour requiring areas 
30 such as the lips, cheeks and eyes. 

A digital image of a customer's face may be taken at a 
point-of-sale department store counter for storage into a 
central memory and then made available at a website. The 
35 image may also be storable locally at the store or within an 
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intranet system of the cosmetic supplier. Also envisioned 
is the alternative possibility for the customer scanning 
their own actual image from their home computer. Many of 
these customers now have relatively inexpensive digital 
5 imaging cameras interfaceable with personal computers and 
thereby can connect directly to the internet. 

Once the natural skin colour has been identified, a 
consultant's choice is automatically determined from a 

10 software stored set of 12-20 colours matching the person's 
skin type. This provides a colour matching facial 

foundation for building the first stage of colour makeover. 
Feedback is then accepted into the program on the customer' s 
own preferences. For instance, these preferences can be 

15 with respect to three possible "looks" such as fashion, 
natural or go-to-work types. 

A lipstick can then be provided as a consultant's best 
choice. The customer can also provide input for her 
20 favourite colour. The pick may be a natural or dramatic 
look. These choices are then fed directly into the program 
for facial display on the monitor. 

Other colour cosmetics can be applied in the same 
25 consultant's choice and consumer preference feedback mode. 
These cosmetics can include eye shadow, eyeliner, lipliner 
and blush. The monitor can display images from no makeup to 
full makeup with all products or any step in between. 

30 When a particular look including all the necessary colour 
cosmetics have been selected, a printer is activated with an 
ordering slip for the selected cosmetics. The customer is 
identified by a number and the selected products given a 
barcode symbol or other designation correlating the product 

35 with the customer. 
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The customer's image can be placed on an internet site. At 
some subsequent time, the customer can view the Internet 
site and from the comfort of their own terminal evaluate 
5 various makeovers on their face. Cosmetics to accomplish 
the actual makeover can then be ordered on the web site or 
through any other channel. 

Equipment for the method includes a digital camera available 
from several sources. Nikon provides a suitable high 
resolution digital camera under the trademark Coolpix 900™ 
providing an image size of 1080 x 960 pixels, VGA mode 640 x 
480. Another digital camera, which may be utilised, is the 
PhotoGenie™ sold with the ePHOTO 1680 camera by the Agfa 
Corporation. Image size is 1280 x 1200 pixels, VGA mode 640 
x 480. Software for operating the system can be obtained 
from Torchlight Corporation of Seattle. Programs upon which 
this software is based are found in U.S. Patent 5,854,850, 
U.S. Patent 5,825,941 and U.S. Patent 5,687,259 all herein 
incorporated by reference. 

The foregoing description illustrates selected embodiments 
of the present invention and in light thereof variations and 
modifications would be suggested to one skilled in the art, 
25 all of which are within the scope of this invention. 



10 



15 



20 



WO 01/04839 



PCT/EP00/05409 



- 7 - 



CLAIMS : 

1. A method for applying a virtual makeover to a person's 
5 face, the method comprising: 

(i) directing a digital camera at a person's face 
to register an image of at least a portion 
thereof; 

(ii) calculating colour parameters on two or more 
10 areas of the image to identify a natural colour 

of the skin; 

(iii) transmitting the image to a monitor for 
displaying the face; 

(iv) correlating the natural skin colour with a 
15 predetermined palette of colours appropriate to 

the calculated natural skin colour; 

(v) locating areas of the face for application of a 
colour cosmetic; 

(vi) displaying the facial image with the 
20 predetermined colour palette on the located 

areas; and 

(vii) optionally placing an order for colour 
cosmetics corresponding to the predetermined 
colour palette. 

25 

2. An apparatus for applying virtual makeover to a person's 
face, the apparatus comprising: 

(i) a digital camera for acquiring information on 
the person's facial image; 
30 (ii) a program for calculating colour on two or more 

different areas of the facial image not 
normally covered by cosmetics to identify a 
natural colour of the person's skin; 

(iii) a monitor for displaying the facial image; 
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(iv) a program for painting a predetermined colour 
palette correlated with the natural skin colour 
onto certain located areas of the face normally 
subject to colour cosmetic treatment; and 
5 (v) a printer delivering a printout of the 

predetermined colour palette. 
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